Papers by Hans Hergen Lehmann
Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities (2026.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used for knowledge-based reasoning tasks, yet understanding when they rely on genuine knowledge versus superficial heuristics remains challenging. |
| Approach: | They propose to ask LLMs to compare numerical attributes to find out which country has the highest population, France or Germany. |
| Outcome: | The proposed model comparisons show that heuristics override principled reasoning for larger models, while smaller models show no discrimination. |